Add Keccak-256 (Ethereum-style, original 0x01 padding) by fabrice102 · Pull Request #3245 · aws/aws-lc

fabrice102 · 2026-05-12T02:33:54Z

Issues:

N/A

Description of changes:

[edit] Following review, Keccak-256 was moved out of the FIPS module into crypto/keccak/ (mirroring blake2), reusing the module's FIPS 202 primitives via external linkage; FIPS202_Init is unchanged. The public API is now EVP_keccak256() (no underscore), tests live in crypto/keccak/keccak256_test.cc, and AWSLC_API_VERSION is bumped to 36. See the comment below for the full changelog — the "Approach" section above describes the original in-module design and is superseded.

Adds Keccak-256 with the original Keccak submission's padding byte (0x01) — the variant used by Ethereum, Solidity's keccak256(), and other blockchain ecosystems. This is not FIPS 202 SHA3-256 (which uses 0x06 padding); rate (1088 bits) and digest length (256 bits) match, but the padding byte differs, so outputs differ for every input.

Approach:

The implementation lives inside the FIPS module
(crypto/fipsmodule/sha/sha3.c) so it shares the audited Keccak-f[1600] permutation, FIPS202_Init/Update/Finalize buffering, and assembly backends with the SHA-3 family. The only delta in the module's algorithmic code path is one extra accepted padding byte:

if (pad != SHA3_PAD_CHAR &&
    pad != SHAKE_PAD_CHAR &&
    pad != KECCAK_PAD_CHAR) {  // 0x01
  return 0;
}

The new public surface is exactly EVP_keccak_256() (plus the existing EVP_Digest* machinery). This matches the BoringSSL/AWS-LC convention established by SHA-3: the EVP_MD is the public API; direct C entry points (KECCAK_256_Init/Update/Final, one-shot KECCAK_256()) live in crypto/fipsmodule/sha/internal.h and are exported only for in-tree consumers, mirroring SHA3_256_*.

Call-outs:

NOT FIPS-approved. KECCAK_256_Final and the KECCAK_256() one-shot intentionally omit FIPS_service_indicator_update_state(), so calling Keccak-256 leaves the per-thread approved-services counter unchanged. Verified by a new row in service_indicator_test.cc::kDigestTestVectors with AWSLC_NOT_APPROVED. The negative invariant is enforced both at runtime (the parameterized EVPMDServiceIndicatorTest.EVP_Digests catches accidental _update_state calls) and structurally (a future reviewer can grep the file and see only "Intentionally no" comments).
No NID, no OID. The Ethereum-style 0x01-padding Keccak is not standardised and OpenSSL has not assigned an OID for it either. The EVP_MD's type is NID_undef, and the digest is registered in nid_to_digest_mapping by name only. As a consequence, EVP_keccak_256() cannot be used with EVP_DigestSign* / EVP_DigestVerify* for RSA-PKCS#1-v1_5 or ECDSA, and EVP_marshal_digest_algorithm will fail with DIGEST_R_UNKNOWN_HASH. objects.txt, obj_dat.h, and nid.h are intentionally untouched.
OpenSSL compatibility. The registered EVP_MD name is KECCAK-256 (uppercase, hyphenated), matching OpenSSL 3.2+'s default-provider registration in providers/implementations/include/prov/names.h. Portable code that does EVP_MD_fetch(NULL, "KECCAK-256", NULL) on OpenSSL works against AWS-LC via EVP_get_digestbyname("KECCAK-256"). A keccak-256 lowercase alias is also registered.
FIPS module impact. The audited Keccak-f[1600] permutation, absorb/squeeze code, and existing SHA-3 entry points are unchanged byte-for-byte. The only edit inside the module boundary is the 3-line FIPS202_Init pad-byte check above plus the new KECCAK_256_* wrappers.

Testing:

All Keccak-256 tests live in crypto/digest_extra/digest_test.cc alongside the rest of the digest tests. New coverage:

kTestVectors[] — 3 well-known Ethereum-ecosystem vectors (empty / "abc" / fox-pangram). These exercise the EVP one-shot, byte-at-a-time streaming, copy/move-on-context, and unaligned-input paths in TestDigest().
DigestTest.Keccak256DiffersFromSHA3_256 — invariant: for the same input, Keccak-256 and SHA3-256 outputs must always differ. Guards against a future maintainer "fixing" the padding byte.
DigestTest.Keccak256KAT — file-driven tests using 137 short-message vectors (0..1088 bits) and 100 long-message vectors (2184..110688 bits), in the same Len/Msg/MD format used by SHA-3 KATs. Mirrors the FileTestGTest pattern from sha3_test.cc::NISTTestVectors.
DigestTest.Getters — extended with EVP_get_digestbyname("KECCAK-256"), the lowercase alias, and an EVP_MD_type == NID_undef assertion.
service_indicator_test.cc::kDigestTestVectors — one new row, parameterized as EVP_Digests/13, asserts AWSLC_NOT_APPROVED through Init / Update / Final.

KAT vectors are generated by util/gen_keccak256_kat.py, which uses pycryptodome (an independent Keccak-256 implementation) as the reference. The script sanity-checks pycryptodome against the well-known Ethereum vectors before generating, so a misconfigured environment fails loudly. Reference and implementation under test come from completely different code, so the test asserts genuine cross-implementation agreement.

bssl speed -filter KECCAK-256 is wired up next to the SHA-3 benchmarks for performance regression tracking.

Verified end-to-end:

Debug build: 2707/2707 active crypto_test cases pass, including DigestTest.Keccak256*, SHA3Test.* (no regression), and the full digest suite.
FIPS build (-DFIPS=1 -DBUILD_SHARED_LIBS=1): 3719/3719 active cases pass, including all 14 rows of EVPMDServiceIndicatorTest.EVP_Digests (was 13).
bssl speed -filter KECCAK-256 produces sensible numbers across all default chunk sizes.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

codecov-commenter · 2026-05-16T08:08:28Z

Codecov Report

❌ Patch coverage is 87.27273% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.40%. Comparing base (f4aaf20) to head (ac4ae36).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
crypto/fipsmodule/sha/sha3.c	78.12%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3245      +/-   ##
==========================================
- Coverage   78.42%   78.40%   -0.03%     
==========================================
  Files         695      695              
  Lines      124087   124144      +57     
  Branches    17226    17236      +10     
==========================================
+ Hits        97321    97334      +13     
- Misses      25845    25886      +41     
- Partials      921      924       +3

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

manastasova

Really clean PR! My only major comment is around the decision to add Keccak256 into the FIPS module. Would it be possible to integrate it outside of the module instead, similar to how blake2 is handled, while reusing the FIPS 202 functions from the FIPS module? This would also mean relocating the test vectors.

manastasova · 2026-05-18T09:19:31Z

    0x85, 0x3c, 0x64, 0xa1, 0x56, 0x6f, 0xeb, 0x76, 0x25, 0x9a, 0x4a, 0x44,
    0x23, 0xf7, 0xcf, 0x46};

+// Keccak-256 of |kPlaintext|, computed by util/keccak_ref.py.


util/gen_keccak256_kat.py ?

fixed in second commit

manastasova · 2026-05-18T09:57:04Z

+}
+
+// |EVP_MD_FLAG_DIGALGID_ABSENT| matches OpenSSL 3.2+'s
+// |PROV_DIGEST_FLAG_ALGID_ABSENT| on KECCAK-{224,256,384,512}. With


Good point. Are there other algorithms that should update the flag EVP_MD_FLAG_DIGALGID_ABSENT?

manastasova · 2026-05-18T10:03:46Z

+    // Keccak-256 has no NID/OID (the Ethereum-style 0x01-padding variant is
+    // not standardised). Registered by name only; matches OpenSSL 3.2+'s
+    // "KECCAK-256" provider name for cross-library lookups.
+    {NID_undef, EVP_keccak_256, "KECCAK-256", "keccak-256"},


Do we need the keccak-256 entry? blake2 has no entry in the table. Should we add it as well?

We need this entry to be able to use EVP_get_digestbyname("KECCAK-256"), which is necessary to make it compatible with OpenSSL 3. OpenSSL 3 does not support EVP_keccak_256().

manastasova · 2026-05-18T10:35:19Z

+// permutation as the SHA-3 family but is NOT FIPS-approved: it does not
+// update the FIPS service indicator. Streaming functions are exposed only
+// internally; public consumers reach Keccak-256 via |EVP_keccak_256|.
+//


nit: extra comment line here

fixed in second commit

manastasova · 2026-05-18T11:14:29Z

       !SpeedHash(EVP_sha3_384(), "SHA3-384", selected) ||
       !SpeedHash(EVP_sha3_512(), "SHA3-512", selected) ||
 #endif
+#if defined(OPENSSL_IS_AWSLC)


Is it not compatible with openssl?

OpenSSL 3 does not support EVP_keccak_256() which this code uses.

manastasova · 2026-05-18T11:30:10Z

@@ -0,0 +1,105 @@
+#!/usr/bin/env python3


Missing license header.

fixed in second commit

manastasova · 2026-05-18T11:34:17Z

+    out_dir = os.path.normpath(out_dir)
+    os.makedirs(out_dir, exist_ok=True)
+
+    # Sanity check pycryptodome's output against the well-known Ethereum


manastasova · 2026-05-18T11:35:09Z

  crypto/fipsmodule/sha/testvectors/SHA3_384ShortMsg.txt
  crypto/fipsmodule/sha/testvectors/SHA3_512LongMsg.txt
  crypto/fipsmodule/sha/testvectors/SHA3_512ShortMsg.txt
+  crypto/keccak/testvectors/KECCAK_256ShortMsg.txt


Alphabetical order would be better

fixed in second commit

- Add missing license header to util/gen_keccak256_kat.py - Define SN_keccak_256/LN_keccak_256 in nid.h and use them in nid_to_digest_mapping instead of raw string literals - Fix incorrect script name in service_indicator_test.cc comment - Remove extra blank comment line in internal.h - Fix alphabetical ordering of keccak test vectors in sources.cmake

fabrice102 · 2026-06-26T20:14:13Z

Thank you for the detailed review @manastasova! I've added a new commit to address your comments.

Re:

My only major comment is around the decision to add Keccak256 into the FIPS module. Would it be possible to integrate it outside of the module instead, similar to how blake2 is handled, while reusing the FIPS 202 functions from the FIPS module?

We would still need to make 2 changes to the FIPS module:

Export the FIPS 202 functions (they are currently not exported).
Modify the FIPS202_Init function to allow for the Keccak padding: https://github.com/fabrice102/aws-lc/blob/e1c7d24fc99a986b5881212ced67da70ad9c92ff/crypto/fipsmodule/sha/sha3.c#L119-L124

Let me know if you prefer doing it that way.

- Add missing license header to util/gen_keccak256_kat.py - Define SN_keccak_256/LN_keccak_256 in nid.h and use them in nid_to_digest_mapping instead of raw string literals - Fix incorrect script name in service_indicator_test.cc comment - Remove extra blank comment line in internal.h - Fix alphabetical ordering of keccak test vectors in sources.cmake

manastasova · 2026-07-14T22:44:35Z

Re:

We would still need to make 2 changes to the FIPS module:

Export the FIPS 202 functions (they are currently not exported).
Modify the FIPS202_Init function to allow for the Keccak padding: https://github.com/fabrice102/aws-lc/blob/e1c7d24fc99a986b5881212ced67da70ad9c92ff/crypto/fipsmodule/sha/sha3.c#L119-L124

Let me know if you prefer doing it that way.

These changes sound reasonable.

Implement the pre-standardization Keccak-256 hash, the variant that uses the original 0x01 padding byte as used by Ethereum and Solidity's keccak256(). It shares the rate, capacity and digest length of FIPS 202 SHA3-256 but differs in the padding byte (0x06), so it produces distinct digests and is not FIPS-approved. The implementation lives outside the FIPS module in crypto/keccak/, following the blake2 template: - crypto/keccak/keccak.c + internal.h provide Keccak256_Init/Update/ Final and the Keccak256 one-shot. - The EVP_MD wrapper EVP_keccak256() lives in digest_extra.c alongside the other non-FIPS digests (EVP_md4, EVP_blake2b256). - It is registered by name only as "KECCAK-256" (plus the "keccak-256" lowercase alias) with NID_undef, matching OpenSSL 3.2+'s provider name so EVP_get_digestbyname() lookups are cross-compatible. Because Keccak-256 has no OID/NID, the names are string literals in the digest table rather than SN_/LN_ macros (nid.h is generated from objects.txt and cannot carry an OID-less entry). It has no OID and must not be used where a digest AlgorithmIdentifier is marshalled. - Tests are in crypto/keccak/keccak256_test.cc; digest_extra keeps only a generic vector row and the getter assertions, as blake2 does. Changes to the FIPS module are limited to giving the shared FIPS 202 buffering primitives (FIPS202_Reset/Update/Finalize) external linkage so crypto/keccak can reuse the audited Keccak-f[1600] permutation and padding logic instead of duplicating it. FIPS202_Init is unchanged and still rejects non-FIPS-202 padding; Keccak256_Init sets up the context with the 0x01 padding byte itself. AWSLC_API_VERSION is bumped to 36 so consumers can feature-detect EVP_keccak256 (v5.2.0 already shipped version 35 without it); the bssl speed benchmark gates its EVP_keccak256 call on AWSLC_API_VERSION >= 36 accordingly. The KAT vectors (KECCAK256ShortMsg.txt / KECCAK256LongMsg.txt) are generated by util/gen_keccak256_kat.py against pycryptodome. The embedded test data in generated-src/crypto_test_data.cc.tar.bz2 was regenerated with util/generate_build_files.py, and the version scripts (crypto/libcrypto.txt/.map) with util/generate_initial_version_scripts.sh. EVP_keccak256 is exported at the AWS_LC_1.0 node.

fabrice102 · 2026-07-20T15:46:04Z

Pushed the refactor implementing the approved direction (move outside the FIPS module, mirror blake2). Summary of what changed since the last review:

Moved Keccak-256 out of the FIPS module — it now lives in a new crypto/keccak/ (keccak.c + internal.h), reusing the module's audited Keccak-f[1600] permutation and FIPS 202 buffering rather than duplicating them, exactly as discussed.

The two FIPS-module changes we agreed on:

FIPS202_Reset/Update/Finalize are given external linkage (de-static'd, declared in sha/internal.h) so the out-of-module code can reuse them. They are not OPENSSL_EXPORTed, so nothing new leaves libcrypto.
FIPS202_Init is reverted — it no longer accepts the Keccak pad and is unchanged from mainline. Instead, Keccak256_Init sets up the 0x01 padding itself. So the module's algorithmic code is now byte-for-byte identical to mainline.

Public API renamed EVP_keccak_256 → EVP_keccak256 (no underscore) for consistency with EVP_shake256 / EVP_blake2b256 and the keccak256 spelling used by Ethereum.

Tests moved to a dedicated crypto/keccak/keccak256_test.cc (like blake2_test.cc); digest_test.cc keeps only the generic vector row + getter assertions, matching how blake2 is handled. The service-indicator test row was removed since Keccak-256 is no longer a module function.

Bumped AWSLC_API_VERSION to 36 so consumers can feature-detect EVP_keccak256 (v5.2.0 shipped 35 without it); the bssl speed gate uses >= 36.

Name registration: since Keccak-256 has no OID/NID, "KECCAK-256"/"keccak-256" are now string literals in the digest table rather than nid.h macros (nid.h is generated from objects.txt and can't carry an OID-less entry) — so objects.txt/nid.h stay untouched and the generated-file CI check passes.

fabrice102 requested a review from a team as a code owner May 12, 2026 02:33

fabrice102 had a problem deploying to manual-approval May 12, 2026 02:34 — with GitHub Actions Failure

manastasova reviewed May 18, 2026

View reviewed changes

justsmth requested a review from jakemas May 27, 2026 13:02

fabrice102 had a problem deploying to manual-approval June 26, 2026 19:24 — with GitHub Actions Error

fabrice102 force-pushed the keccak-256 branch from e1c7d24 to 9f128b1 Compare July 10, 2026 17:34

fabrice102 had a problem deploying to manual-approval July 10, 2026 17:34 — with GitHub Actions Error

fabrice102 had a problem deploying to manual-approval July 10, 2026 19:44 — with GitHub Actions Error

fabrice102 had a problem deploying to manual-approval July 10, 2026 19:45 — with GitHub Actions Error

fabrice102 force-pushed the keccak-256 branch from 13a9d13 to ac4ae36 Compare July 14, 2026 20:41

fabrice102 had a problem deploying to manual-approval July 14, 2026 20:41 — with GitHub Actions Error

fabrice102 force-pushed the keccak-256 branch from ac4ae36 to 2f9e81d Compare July 20, 2026 15:44

fabrice102 had a problem deploying to manual-approval July 20, 2026 15:45 — with GitHub Actions Error

fabrice102 requested a deployment to manual-approval July 20, 2026 15:46 — with GitHub Actions Waiting

Uh oh!

Conversation

fabrice102 commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issues:

Description of changes:

Approach:

Call-outs:

Testing:

Uh oh!

codecov-commenter commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

manastasova left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fabrice102 commented Jun 26, 2026

Uh oh!

manastasova commented Jul 14, 2026

Uh oh!

fabrice102 commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fabrice102 commented May 12, 2026 •

edited

Loading

codecov-commenter commented May 16, 2026 •

edited

Loading

manastasova left a comment •

edited

Loading